Proc AMS, to appear. 3 May 2008. 



A LITTLEWOOD-RICHARDSON RULE FOR GRASSMANNIAN 

'66 : PERMUTATIONS 
O 
O 

CN ; KEVIN PURBHOO AND FRANK SOTTILE 

>>: 

^ ■ Abstract. We give a combinatorial rule for computing intersection numbers on a flag 

manifold which come from products of Schubert classes pulled back from Grassmannian 
• projections. This rule generalizes the known rule for Grassmannians. 

o 
u 

. One of the main open problems in Schubert calculus is to find an analog of the Little- 

wood- Richardson rule for fiag manifolds [StaOOl Problem 11], and more generally to find 
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combinatorial formulae for intersection numbers of Schubert varieties. This problem was 
CN ■ recently solved by Coskun for two-step fiag manifolds |Co07j . 

We give such a combinatorial interpretation for intersection numbers of Grassmannian 
(yQ I Schubert problems on any type A fiag manifold. This number counts certain objects 

' that we call filtered tableaux which satisfy conditions coming from the Schubert problem. 

When the fiag manifold is a Grassmannian this coincides with a standard interpretation 
^ . of these numbers obtained from the Littlewood-Richardson rule. Grassmannian Schubert 

f — I problems on the flag manifold were studied in |RSSS06j : they are exactly the Schubert 

O ■ problems which appear in the generalization of the Shapiro conjecture to flag manifolds 

given there. 

^ ! In Section [T] we deflne flltered tableaux, give an example, and state our formula, which 

^ I we prove in Section [2J Our proof uses some identities of |BS98j which were established 

using geometry, and is thus not completely combinatorial. In Section [3] we explain how 
our formula relates to one coming from Monk's formula |Mon59] and discuss how to give 



a purely combinatorial proof based on the rule of Kogan [KogOl . 

We thank the Centre de recherches mathematiques in Montreal and the organizers of 
the workshop on Combinatorial Hopf Algebras and Macdonald Polynomials in May 2007 
where this collaboration was begun. We also thank Hugh Thomas for helpful discussions, 
and Chris Hillar for comments on the manuscript. 



1. A Littlewood-Richardson rule for Grassmannian Schubert problems 

For background on flag manifolds and Schubert calculus, see |Ful97] . We flx a posi- 
tive integer n throughout. Let a = {ai,a2, . • • ,am} be a non-empty subset of [n—1] := 
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{1,2,..., n— 1}, which we write in increasing order 

a : = ao < tti < ■ ■ ■ < am < ctm+i = n . 
A partial flag of type a is a sequence F, of hnear subspaces in C" 

F, : {0} C Fi C F2 C ■ ■ ■ C F„, C C" , 
where dimFj = ttj. The set J-'£a of all flags of type a is a complex manifold of dimension 

m 

dim(a) := y^(n - ai){ai - aj-i) . 

i=l 

Schubert varieties and classes in Ma are indexed by permutations w of {1,2, . . . ,n} whose 
descent set is contained in a. For a permutation w, let be the class of the Schubert 
variety corresponding to w, following the conventions in |Ful97j . Its cohomological degree 
is 2£{w), where £{w) counts the number of inversions {i < j \ w{i) > w{j)} of w. 

If /? C a is another subset then there is a projection tTq,^^ : Ma —>■ Mp whose fibres are 
products of flag varieties. When (3 = {b} is a singleton, Tip is the Grassmannian Gr(6, n) 
of 6-planes in C^. In this case, we write iTh for tTq,^^. We note that 'n'a/sO'w is just the 
Schubert class cr^ G H*{Mp). 

Schubert classes in Gr(6, n) are also indexed by partitions A, which are northwest-justified 
arrays of boxes in a 6 x [n — b) rectangle, □{,. Associated to a partition A is the Grass- 
mannian permutation w with shape A and descent at b. This permutation has a unique 
descent at b, and its first b values are 

w{i) = i + \{b + 1 — i) for i = l,...,b. 

Here, X{i) denotes the number of boxes in row i of A. We write ax for the Grassmannian 
Schubert class a^. Here are three partitions with 6 = 3 and n = 7; the third is also drawn 
inside They correspond to the Grassmannian permutations 1352467, 1372456, and 

2471356. 

m rrm rrrn rrm 



Let |A| be the number of boxes in A. This is half the cohomological degree of the Schubert 
class ax and is the complex codimension of the associated Schubert variety. 

The Littlewood-Richardson rule for the Grassmannian expresses a product ax ■ a^ of two 
Schubert classes as a sum of classes ay where \, ji <Z v with = + |A|. In this rule, 
the coefficient c^l^ of a^ is the number of Littlewood-Richardson tableaux of skew shape 
v/ IX := V — ^ and content A. These are fillings of the boxes in v/ ^ with positive integers 
such that 

(i) The entries weakly increase left-to-right across each row and strictly increase down 
each column. 

(ii) The number of js in the filling is equal to A(j), the number of boxes in row j of A. 

(iii) If we read the entries right-to-left across each row and from the top row to the bot- 
tom row, then at every step we will have encountered at least as many occurrences 
of i as of i+1 for each positive integer i. 
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For example, here are some Littlewood-Richardson tableaux. 
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A Gmss7nannian Schubert class in the cohomology ring of J-'ia is the puUback of a 
Schubert class along a projection to a Grassmannian. That is, it has the form vt^cta where 
b E a and A C These are indexed by pairs (6, A) with A C I 1^. 

A Grassmannian Schubert problem is a list ((ai, Ai), . . . , (a^, A^)) with ai < ■ ■ ■ < a^. 
We require that for every i = 1 , . . . , s we have Oj G a and A j C , and also 



(1.2) |Ai| + IA2I + ■■■ + |A,| 

By the dimension condition (11.2p . we have 



dim(a) . 



i=l 



H2dim{a)^jrQ = Z ■ [pt]. 



where [pt]a is the class of a point in !Fia- The problem that we solve is to give a combi- 
natorial formula for the coefficient of [pt]a in this product. Note that if a 2 {^i, • • • , cig} 
this coefficient is zero (e.g. by [KnuOOl Lemma 1]), and so we will generally assume that 
a = {ai, . . . ,aj. 

Write for the union of all rectangles for each a G a, where the rectangles all 
share the same upper right corner. Here are three such shapes when n = 7. 

m: 



J235 



J 145 



A shape fi C is a subset of boxes which are northwest justified. For example, when 
n = 6, the shaded boxes are four shapes in ^231- 



Definition 1.1. Let A = ((ai, Ai), . . . , (a^. As)) be a Grassmannian Schubert problem. Set 
a = {ai, 02, ... , tts} and fix a shape fi C A filtered tableau T, with shape ji and content 
A is a sequence 

/i, : = /io C /ii C /i2 C ■ ■ ■ C lis+i C /is = /i 

of shapes together with fillings Ti, . . . ,Ts of the skew shapes by positive integers 

which satisfy the following properties. 

(1) The skew shape must fit entirely within the rectangle □(!. C '^q,. 

(2) The filling Tj is a Littlewood-Richardson tableau of content Aj. 

Note that we must have = |Ai| -|- ■ ■ ■ -|- |As|. ^ 
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An induction shows that the coefficient of [pt]f, = ctq^ in a product ctai ■ ■ ■ cta^ in 
H*{GT{b,n)) is the number of filtered tableaux with shape whose content is the se- 
quence ((6, Ai ),..., (6, Xs)). We generalize this to any flag manifold. 

Theorem 1.2. Let A = ((ai, Ai), . . . , (a^, A^)) be a Gmssmannian Schubert problem on 
Tla. Then the coefficient of [pt]a in the product TT^-CrXi is the number of filtered tableaux 
with shape and content A. 

Example 1.3. We use this formula to compute the intersection number A^, defined by 
N[pt]a = <(cTnn) ■ vr;((Tnn) ■ T^sicTQi) ■ 7tI{(t ) ■ 7r;((T|) . 

Here, a = [4] and is the full staircase shape. There are exactly three sequences of 
shapes /i, which satisfy the condition (1) in the definition of filtered tableaux. 
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Each of the ffist two sequences support a unique ffitered tableau satisfying condition (2), 
while the third supports two; thus the required intersection number is 4, which may be 
verified by direct computation using the Fieri formula for flag manifolds |Sot96j . Indeed, 
there is a unique Littlewood- Richardson tableau of shape u/fi and content A when A is 
a single row or column and also when the shapes of u/fi and A are the same or rotated 
by 180°. The only skew shape here which admits more than one Littlewood-Richardson 
tableau is when A = P and i^/ fJ^ = cfP. There are two such Littlewood-Richardson tableaux, 
given in (II. ip . and this occurs in the middle of the third chain. ^ 



2. Proof of Theorem 11.21 

Let J-'i := J-'£[n-i] be the manifold of complete flags in C", which has dimension Q). 
Its Schubert classes are indexed by all permutations w of the numbers {1, 2, . . . , n}. We 
prove a strengthening of Theorem 11.21 for the full flag manifold and use this to deduce 
Theorem 11.21 for all partial flag manifolds. We give the key definition of this section. 

Definition 2.1. A permutation i;; is a valley permutation with floor at a if 

w{l) > w{2) > ■ ■ ■ > w{a) and w{a+l) < w{a+2) < ■ ■ ■ < w{n) . ^ 
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For example, 531246 and 643125 are valley permutations with floor at 3. We associate 
a shape fi = fi{w) to any valley permutation w. If w has floor at a, then fi{w) is the shape 
whose rows are 

w{l)-l > w{2)-l > ■■■ > w{a)-l > 0. 

This has either a or a— 1 rows. Observe that w is determined by fi{w) and that £{w) = 
\n{w)\ where i{w) counts the inversions in w. For example, 

/i(531246) = I I I I I and /i(643125) = [ 



] 



Theorem 2.2. Let A = ((ai, Ai), . . . , (at, \t)) with ai < a2 < ■ ■ ■ < at and suppose that w 
is a valley permutation with shape /i. Then the coefficient of in the product ni=i '^li^M 
in the cohomology ring of J^i is the number of filtered tableau with shape fi and content A. 

Since the class [pt] of a point in H*{J-'i) is indexed by the longest permutation, which is 
a valley permutation with shape ^[„_i]. Theorem 12.21 implies Theorem 11.21 for M[n-i]- We 
deduce Theorem 11.21 for general flag manifolds from the case for Ti^n-i]- 

Proof of Theorem \1.2 . Suppose that b ^ a, say < b < Oj+i, and set a' := aU {b}. We 
assume that the theorem holds for Ma', and deduce it for Ma- 
Let K, be the rectangular partition with b—ai rows and ttj+i— 6 columns. Set A' : = 
((ai, Ai), . . . , (6, k), . . . , {os, Xs))- Note that tt*, ,,cr«; is dual to 7r*,^^[pt]a in H*{Ma') under 
the Poincare pairing. Thus, for any r G H*{J-'ia) we have 

where [[ptj^jr denotes the coefficient of [pt]a in r. In particular, 

(2.1) [[pt]a'] n = [[pt]"] n <'^^'- 

{a,A)eA (a',A')GA' 

There is a bijection between filtered tableaux with shape '^q, and content A and those with 
shape '^a' and content A', obtained by inserting the unique Littlewood- Richardson tableau 
of shape and content k into the filtration. Thus counting either set of filtered tableaux 
gives the coefficient fl2.ip . ^ 

A Schubert class appears in a product (Ju ■ ■ ■ a^, of Schubert classes if, when we expand 
the product in the basis of Schubert classes, appears with a positive coefficient. 

We will prove Theorem 12.21 by induction on the number of terms t in the product. 
Important for this is the following proposition which summarizes some discussion at the 
beginning of Section 1 in |BS98j . 

Proposition 2.3. // a Schubert class appears in the product a^-nlax, then the following 
conditions hold. 

(1) Whenever i < a < j , we have w{i) > v{i) and w{j) < v{j). 

(2) If i < j < a and v{i) < v{j), then w{i) < w{j). If a < i < j and v{i) < v{j), then 
w{i) < w{j). 

In |BS98] , it is shown that the conditions in Proposition 12.31 define an order relation 
V <a w, which is a suborder of the Bruhat order. We deduce an important lemma. 
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Lemma 2.4. If cXw appears in n!=i^a,;^Aj then w has no descents after position at- 

Proof. We prove this by induction on t. It holds when t = 0, as the multiphcative identity 
in cohomology is the Schubert class indexed by the identity permutation. 

Suppose that a.^, appears in the product n!=i ^a^'^Ai- Then there is some permutation v 
such that o"t, appears in the product ni=i '^ai'^>^i ^'^^ '^w appears in the product ■ rtl^crxf 
Hence v <at w. Since v has no descents after position at-i and at-i < at, condition (2) of 
Proposition 12.31 implies that w has no descents after position at. ^ 

For permutations v,w and a partition A C Ella, let c^ax be the coefficient of in the 
product (T„ ■ 7r*crA. One of the main results in |BS98j is the following identity. 

Proposition 2.5. Suppose that v <a w and x <a z with wv~^ = zx~^ . Then for every 
X^niawe have = c^^ ^ 

Suppose that a shape z/ C ^[n-i] has either 6—1 or b rows. We define z/|{, to be the 
intersection of the shape u with I \h. 

Proof of Theorem \2.B . We proceed by induction on t. The theorem holds (trivially) for 
t = 0; assume that t > and that it holds for t — 1. 

Let W7 be a valley permutation with shape /i, and suppose that w appears in the product 
n!=i '^li'^Xi- Then by Lemma [2^ w has a fioor at at. Let us expand the product 

t-i 

i=l V 

Then the coefficient of cr^^, in the product IlLi '^a.i^^i 



V „w 

v,at,\t ■ 



Suppose that v <at w. Since w has a floor at at, Proposition 12.3( 2) implies that 

v{l) > v{2) > ■ ■ ■ > v{at). 

If the coefficient c" ^ 0, so that v can contribute to this sum, then Lemma 12.41 implies 
that V has no descents after position Ot-i. Since a^— 1 < Ot-i < at, this implies that f is a 
valley permutation with a floor at at. 

Let u be the shape of v. Since both w and v have floor at at, both ^ and u have either 
at— 1 or at rows, and thus /i/z^ C UJat- The theorem would follow if we knew that 

(2-2) = cf . 

To see this, note that there is a bijection between flltered tableaux on fi with content 
((ai, Ai), . . . , {at, Xt)) and triples {u, T,, T) where z/ C /x, T, is a flltered tableau of shape u 
and content ((oi, Ai), . . . , (at-i, Xt-i)), and T is a Littlewood-Richardson tableau of shape 
II /u and content A; hence the number of these is 

c ■ cv 

At 

V<a-fW 
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But (12. 2p follows from Proposition 12.51 Let x (respectively z) be the permutation ob- 
tained from V (respectively from w) by reversing the first at values, i.e. 



v{at + 1 — i) if 1 < z < at 
v(i) otherwise. 



at ) 



Then x and z are Grassmannian permutations with descent at, and shapes z/|a^ and fi 
respectively, and fJ^/i^ = {f^laj / {^\at) ■ Furthermore, x z and wv~^ = zx~^, from which 
we deduce ([22]) • ^ 

3. Further Remarks 

When all the classes have degree 2 (A, = □, a single box), the multiplication formula 
■ 7r*,n is due to Monk jMon59j . Monk's formula states that 



(3.1) cT^-Kn = Yl 



j<i<k 

where rjk G Sn is the transposition swapping j and k. Iterating Monk's formula one sees 
that the coefficient of [pt]a in a product nf™*""'' ^a,n is obtained by counting certain chains 
in the Bruhat order. It is not hard to see directly from (13.11) that each permutation w in 
such a chain corresponds to a shape fi in such that the number of boxes in the column 
j of /i equals G [j] \ w{k) > w{j + 1)}, for all j G {min(a), . . . ,n — 1}. Indeed, if 
the permutation w does not correspond to a shape, then no term on the right hand side 
of (13. ip corresponds to a shape. It follows that the coefficient is the number of chains of 
shapes in where the ith step involves adding a box in \I3ai , which is the answer given 
by our formula. 

For example, we have 

as there are two chains of shapes which satisfy this condition. 

■ c H c H c 



c c c 



It is possible to give a purely combinatorial proof of Theorem 11.21 using Kogan's for- 
mula [KogOl Theorem 2.4]. This rule is based on insertion of RC-graphs and gives the 



coefficient c^^x^ when v{a+l) < v{a+2) < ■ ■ ■ < v{n). In particular, this gives a formula 
for the product when v and w are a valley permutations with a floor at a, and so we may 
use this in a formula for the intersection numbers of Theorem 11.21 to give a combinatorial 
proof. 



The conventions in Kog01| for Schubert classes differ from those used in this article. 



To compare conventions, it is necessary to replace our permutations w hy w = WqWWq 
throughout. In particular, a cohomology class indexed by w in this article is the class 
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indexed by w in |Kog01| . Thus our condition on v becomes v{l) < v{2) < ■ ■ ■ < v{a), 



which is the condition found in KogOl 



To deduce Theorem 11.21 from this formula, we would need to show that, for valley 
permutations w, v with floor at a, Kogan's rule for c^^ ^^ coincides with the Littlewood- 

Richardson rule for c^^*^, where u = /i(w)|a and /i = iJ,{w)\a- Here, is the shape of v 
and is the shape of w. While this is certainly possible, we chose not to pursue this. 



Appendix A. More examples 
Example A.l. Consider the following product in ^235, 



By Theorem II. 2[ the coefficient of [pt] is the number of filtered tableau with content 
((2, m), (2, (3, ■), (3, P), (5, □), (5, |), (5, |)), which is 18: 
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Example A. 2. We remarked 
in Section [3] that, when ev- 
ery partition is a single box 
(Aj = □), a filtered tableau is 
a particular saturated chain of 
shapes in ^q,. When n = 6 we 
look at this for the problem 

in J^£234- 

To the right is the poset of 
shapes n in '^234, where at level 
t (from the top) the shape has 
at most at and at least a*— 1 
rows. 

Further to the right, we 
count the number of chains in 
this poset, which shows that 
the intersection number is 262. 
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